Skip to content

[Fix] 자소서 분석 개선 예시 문장 생성 방식 수정#81

Merged
whc9999 merged 5 commits into
mainfrom
fix/#24-analysis-prompt
May 26, 2026
Merged

[Fix] 자소서 분석 개선 예시 문장 생성 방식 수정#81
whc9999 merged 5 commits into
mainfrom
fix/#24-analysis-prompt

Conversation

@whc9999
Copy link
Copy Markdown
Collaborator

@whc9999 whc9999 commented May 26, 2026

✨ 어떤 이유로 PR를 하셨나요?

  • feature 병합
  • 버그 수정(아래에 issue #를 남겨주세요)
  • 코드 개선
  • 코드 수정
  • 배포
  • 기타(아래에 자세한 내용 기입해주세요)

📋 세부 내용 - 왜 해당 PR이 필요한지 작업 내용을 자세하게 설명해주세요

  • LLM 프롬프트에서 improvement를 완성된 개선 예시 문장으로 생성하도록 지시 강화
  • improvement에 첨삭 조언이나 지시문이 들어오지 않도록 금지 규칙 추가
  • 지시문 형태의 improvement가 반환되면 빈 값으로 정규화하도록 방어 로직 추가
  • improvement 정규화 테스트 추가

📸 작업 화면 스크린샷

⚠️ PR하기 전에 확인해주세요

  • 로컬테스트를 진행하셨나요?
  • 머지할 브랜치를 확인하셨나요?
  • 관련 label을 선택하셨나요?

🚨 관련 이슈 번호 [#24 ]

Summary by CodeRabbit

  • Improvements
    • Cover-letter analysis now requires improvement suggestions to be full Korean declarative replacement sentences (preferably with metrics/results/actions); instruction-like or directive phrasing is detected and normalized to blank.
  • Tests
    • Added tests ensuring declarative improvements are preserved while instruction-like, spaced, or formal-directive suggestions are normalized/filtered.

Review Change Stack

- LLM 프롬프트에서 improvement를 완성된 개선 예시 문장으로 생성하도록 지시 강화
- improvement에 첨삭 조언이나 지시문이 들어오지 않도록 금지 규칙 추가
- 지시문 형태의 improvement가 반환되면 빈 값으로 정규화하도록 방어 로직 추가
- improvement 정규화 테스트 추가
@whc9999 whc9999 self-assigned this May 26, 2026
@whc9999 whc9999 added the fix label May 26, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 26, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 45488989-087c-47d0-9149-03ba85e42c8b

📥 Commits

Reviewing files that changed from the base of the PR and between 612e815 and ebd881b.

📒 Files selected for processing (1)
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java

📝 Walkthrough

Walkthrough

The PR tightens the AI prompt to require fully formed Korean replacement sentences for per-question "improvement" entries, adds service-side normalization to blank instruction-like LLM outputs, and extends tests to assert both blanking and preservation behaviors.

Changes

Improvement field normalization

Layer / File(s) Summary
Prompt refinement for improvement field
src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java
The prompt's improvement field description and rules now require a fully formed sentence that can replace the extracted sentence, forbids directive phrasing, and includes good/bad examples favoring Korean declarative sentences with metrics/results/actions.
Improvement normalization in analysis service
src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java
Added normalizeImprovement() and isInstructionLikeImprovement(); QuestionAnalysis construction now uses normalizeImprovement(...) to trim input and blank out instruction-like improvements (suffix checks and regex patterns).
Test coverage for improvement normalization
src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java
New tests stub analysisAiClient.analyze(...) for instruction-like, spaced-instruction-like, formal-instruction, and declarative improvements, asserting blanking or preservation as appropriate.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • JobDri-Developer/BackEnd#61: Both PRs modify the cover-letter analysis pipeline, specifically the improvement field semantics in AnalysisAiClient's prompt and AnalysisService's handling of per-question improvements.

Poem

🐰 I hop through prompts that once gave orders clear,
I trade commands for sentences you can paste right here.
The service trims the pleas and keeps the plain fix,
Tests double-check the change with tidy little tricks.
Hooray — a polished line, now ready to appear!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: fixing the generation method of improvement example sentences in cover letter analysis, which aligns with all three modified files.
Description check ✅ Passed The description covers all required template sections with relevant details: reason for PR (code improvement/fix), detailed explanation of changes including prompt enhancements, normalization logic, and tests, and all pre-merge checklist items completed.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/#24-analysis-prompt

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java`:
- Around line 137-141: Update the prompt text in AnalysisAiClient to exactly
mirror the service-layer validation in
AnalysisService.isInstructionLikeImprovement(): include both verb forms like
"추가하세요" and "추가하여" (and other variants the service checks), and list the
sentence-ending patterns the service rejects (e.g., "하세요.", "해주세요.", "해야 합니다.")
so the LLM guidance and the validation logic are consistent; reference the
method AnalysisService.isInstructionLikeImprovement() when locating the exact
phrase/regex patterns to sync and modify the bullet points in the prompt block
accordingly.

In
`@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java`:
- Around line 234-242: The isInstructionLikeImprovement method currently uses
brittle contains() checks that cause false positives; replace those substring
checks with stricter pattern matching (e.g., use a regex to detect
imperative/instructional sentence forms at boundaries or combine imperative
verbs with imperative endings) or invert to a whitelist of declarative endings;
specifically, modify isInstructionLikeImprovement to stop using
improvement.contains("추가하여") / "명확히 하세요" / "보완하세요" and instead use a regex that
matches instruction-like forms (imperative phrases ending with "하세요", "해주세요",
"하세요\\." etc. anchored to sentence boundaries) or validate declarative endings
such as "습니다.", "했습니다.", "됩니다." to avoid discarding valid declarative
improvements referenced by AnalysisAiClient prompt.

In
`@src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java`:
- Around line 239-264: Add the new test method
analyzePreservesDeclarativeImprovement (as in the suggested snippet) to
AnalysisServiceTest to assert that a legitimate declarative improvement string
is preserved; then update the isInstructionLikeImprovement(...) logic (the
method currently using contains("추가하여") and similar checks) to be more
discriminating—e.g., detect imperative forms or explicit instruction markers
(use regex that matches sentences starting with imperative verbs or ending with
imperative/formal instruction endings like "하세요", "해라", "해주세요") instead of
blanket contains checks—so declarative sentences such as the provided
validImprovement are not normalized to empty.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 5c97abe4-5b0f-49ae-9faa-60959cccaf24

📥 Commits

Reviewing files that changed from the base of the PR and between 7de465c and 10f6c03.

📒 Files selected for processing (3)
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java
  • src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java

Comment on lines +239 to +264
@Test
@DisplayName("LLM improvement가 첨삭 지시문이면 빈 값으로 저장한다")
void analyzeNormalizesInstructionLikeImprovement() {
User user = saveUser("analysis-instruction-improvement@example.com");
MockApply mockApply = saveMockApply(user);
Question question = saveQuestion(mockApply, "문제 해결 경험", "저는 로그를 확인하고 쿼리 실행 계획을 분석했습니다.");
when(analysisAiClient.analyze(any(), any())).thenReturn(new AnalysisLlmResponse(
64,
70,
55,
67,
"개선 예시 문장 검증입니다.",
List.of(new AnalysisLlmResponse.QuestionAnalysisItem(
question.getId(),
"저는 로그를 확인하고 쿼리 실행 계획을 분석했습니다.",
"mentioned",
"성과 수치가 부족합니다.",
"성과 수치를 추가하여 문제 해결의 효과를 명확히 하세요."
))
));

AnalysisResponse response = analysisService.analyze(user, mockApply.getId());

assertThat(response.questions().get(0).analyses()).hasSize(1);
assertThat(response.questions().get(0).analyses().get(0).improvement()).isEmpty();
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win

Expand test coverage to include false positive scenarios.

The current test validates that instruction-like improvements are correctly filtered, but doesn't verify that legitimate declarative sentences are preserved. Given the contains() checks in isInstructionLikeImprovement(), you should add test cases to ensure valid improvement sentences aren't incorrectly normalized to empty strings.

🧪 Suggested additional test case
`@Test`
`@DisplayName`("완성된 평서문 improvement는 그대로 저장한다")
void analyzePreservesDeclarativeImprovement() {
    User user = saveUser("analysis-declarative-improvement@example.com");
    MockApply mockApply = saveMockApply(user);
    Question question = saveQuestion(mockApply, "문제 해결 경험", "저는 로그를 확인했습니다.");
    String validImprovement = "저는 로그를 분석하고 누락된 인덱스를 추가하여 응답 시간을 1.8초에서 0.6초로 단축했습니다.";
    when(analysisAiClient.analyze(any(), any())).thenReturn(new AnalysisLlmResponse(
            70,
            75,
            65,
            70,
            "평서문 검증입니다.",
            List.of(new AnalysisLlmResponse.QuestionAnalysisItem(
                    question.getId(),
                    "저는 로그를 확인했습니다.",
                    "mentioned",
                    "성과가 부족합니다.",
                    validImprovement
            ))
    ));

    AnalysisResponse response = analysisService.analyze(user, mockApply.getId());

    assertThat(response.questions().get(0).analyses()).hasSize(1);
    assertThat(response.questions().get(0).analyses().get(0).improvement())
            .isEqualTo(validImprovement);
}

This test would currently fail due to the overly broad contains("추가하여") check.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java`
around lines 239 - 264, Add the new test method
analyzePreservesDeclarativeImprovement (as in the suggested snippet) to
AnalysisServiceTest to assert that a legitimate declarative improvement string
is preserved; then update the isInstructionLikeImprovement(...) logic (the
method currently using contains("추가하여") and similar checks) to be more
discriminating—e.g., detect imperative forms or explicit instruction markers
(use regex that matches sentences starting with imperative verbs or ending with
imperative/formal instruction endings like "하세요", "해라", "해주세요") instead of
blanket contains checks—so declarative sentences such as the provided
validImprovement are not normalized to empty.

- improvement 지시문 판별 로직에서 과도한 contains 조건 제거
- 명령형/지시문 어미 중심으로 improvement 필터링 기준 수정
- 정상적인 평서문 개선 예시가 빈 값으로 정규화되지 않도록 보완
- LLM 프롬프트의 improvement 금지 표현을 서버 검증 기준과 일치하도록 수정
- 완성된 평서문 improvement 보존 테스트 추가
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java (1)

235-241: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Cover spaced imperative form (해 주세요) in normalization.

Current checks catch 해주세요. but can miss 해 주세요. variants, allowing instruction-like text through.

Suggested patch
 private boolean isInstructionLikeImprovement(String improvement) {
     return improvement.endsWith("하세요.")
             || improvement.endsWith("해주세요.")
             || improvement.endsWith("해야 합니다.")
             || improvement.endsWith("필요합니다.")
+            || improvement.matches(".*해\\s*주세요\\.?$")
             || improvement.matches(".*[을를]\\s+(추가|보완|수정)하(세요|십시오).*")
             || improvement.matches(".*명확히\\s+(하|해)(세요|야 합니다).*")
-            || improvement.matches("^(추가|보완|수정).*(하세요|해주세요)\\.$");
+            || improvement.matches("^(추가|보완|수정).*(하세요|해\\s*주세요|해주세요)\\.$");
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java`
around lines 235 - 241, The current imperative-detection expression in
AnalysisService (the return statement that checks improvement.endsWith(...) and
improvement.matches(...)) misses spaced variants like "해 주세요" or "해 주십시오";
update those checks to accept optional whitespace between "하" and the following
honorific/ending. Specifically, add endsWith("해 주세요.") and endsWith("해 주십시오.")
(or use a single endsWith check with normalized whitespace), and change the
regex fragments "하(세요|십시오)" and "하(세요|야 합니다)" to allow optional spaces (e.g.,
"하\\s*(세요|십시오)" and "하\\s*(세요|야 합니다)"), and similarly update any other matches
that assume no space so the method in AnalysisService correctly flags both
joined and spaced imperative forms.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In
`@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java`:
- Around line 235-241: The current imperative-detection expression in
AnalysisService (the return statement that checks improvement.endsWith(...) and
improvement.matches(...)) misses spaced variants like "해 주세요" or "해 주십시오";
update those checks to accept optional whitespace between "하" and the following
honorific/ending. Specifically, add endsWith("해 주세요.") and endsWith("해 주십시오.")
(or use a single endsWith check with normalized whitespace), and change the
regex fragments "하(세요|십시오)" and "하(세요|야 합니다)" to allow optional spaces (e.g.,
"하\\s*(세요|십시오)" and "하\\s*(세요|야 합니다)"), and similarly update any other matches
that assume no space so the method in AnalysisService correctly flags both
joined and spaced imperative forms.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 592478b2-7dff-4d37-b500-3540ea73599c

📥 Commits

Reviewing files that changed from the base of the PR and between 10f6c03 and 85dd5d1.

📒 Files selected for processing (3)
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java
  • src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java

- improvement 지시문 감지 로직에 띄어쓰기 포함 표현 추가
- 해 주세요, 해 주십시오 형태의 지시문 어미를 필터링하도록 수정
- 명령형 정규식에서 하세요/하십시오 앞 공백을 허용하도록 보완
- 띄어쓰기 포함 첨삭 지시문 정규화 테스트 추가
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java`:
- Around line 237-242: The current normalization/filtering logic in
AnalysisService.java for the variable improvement misses imperative variants
ending with "하십시오" and "해주십시오", so phrases like "문장을 명확히 하십시오." or "수정해주십시오."
bypass the filter; update the conditional that checks improvement (the branch
with improvement.endsWith(...) and improvement.matches(...)) to include explicit
checks for "하십시오" and "해주십시오" variants and extend the regex patterns (e.g., the
matches that handle "해 주십시오", "해주십시오", and "하십시오" forms) so those imperative
forms are normalized/filtered the same as existing "해 주세요"/"해 주십시오"/"해야 합니다"
cases.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d5b1bdfe-dad2-49bc-a1fa-01e19d1132f9

📥 Commits

Reviewing files that changed from the base of the PR and between 85dd5d1 and e11ccd6.

📒 Files selected for processing (2)
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java
  • src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java

Comment thread src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java Outdated
- 하십시오, 해주십시오 형태의 격식체 지시문 필터링 추가
- 명확히 하십시오 등 누락된 명령형 표현 정규식 보완
- 격식체 첨삭 지시문이 빈 improvement로 저장되는 회귀 테스트 추가
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java`:
- Line 245: The current improvement.matches(...) call in AnalysisService.java
forces a trailing period via "\\.$", causing imperative commands without
punctuation (e.g., "수정해주십시오") to bypass the filter; update the regex used in the
improvement.matches(...) expression so the final period is optional (for example
by using "(\\.|$)" or making the final "\\." optional) so both forms with and
without a trailing dot match; adjust only the regex string in the
improvement.matches(...) invocation to enforce this change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e394f600-d11e-4e48-afcc-4280c80ea119

📥 Commits

Reviewing files that changed from the base of the PR and between e11ccd6 and 612e815.

📒 Files selected for processing (2)
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java
  • src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java

Comment thread src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java Outdated
- improvement 지시문 정규식의 마침표 조건을 선택 처리
- 수정해주십시오 / 수정해주십시오. 형태 모두 필터링되도록 수정
@whc9999 whc9999 merged commit e50514d into main May 26, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant